Skip to content

nutanix AHV support #3311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

nutanix AHV support #3311

wants to merge 1 commit into from
+28 −0

Conversation

tchernomax
Copy link

add nutanix AHV support

ignition support nutanix AHV, the only thing to do is to add flatcar.oem.id=nutanix in the kernel cmdline.
However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

I managed to create a nutanix AHV flatcar image (without nutanix guest tools... but that doesn't forbid the VM to run) by following this path:

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin

(it tested it on a Nutanix AHV infra, also tried a flatcar update (via nebraska) -> it works)

And creating the VM with the ignition config provided as "cloud_init" "guest_customization":

...
        "guest_customization": {
          "cloud_init": {
            "user_data": "<base64 ignition conf>"
          }
          "is_overridable": false
        }
...

https://www.nutanix.dev/api_reference/apis/prism_v3.html#tag/vms/paths/~1vms/post

I tried to reproduce this path in this PR, but I can't try the resulting image myself.
It build on my desktop, but I can't upload it to a nutanix AHV platform due to my poor home network connection.
And I failed to build the image on the remote host (where I updated the flatcar_production_image.bin.bz2 official image).

How to use

If what I changed in the code is in fact the equivalent of the adding oem_id="nutanix" in the grub.cfg of the OEM partition.
This PR should be ok.

Otherwise, please guide me (I am a noob in flatcar dev).

Testing done

I created an image :

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin
  • upload it to a nutanix ahv cluster
  • created multiple hosts using this image via terraform restapi provider (the nutanix official provider is currently unable to create VM with ignition configuration as "cloudinit" guest info).
  • created a k8s cluster on thoses hosts via rke
  • tried an update via nebraska (flatcar downloaded the new image, I reboot the hosts, it booted on the new version).

I tried this ignition config:

{"ignition":{"version":"3.3.0"},"storage":{"files":[{"path":"/toto","contents":{"compression":"","source":"data:,toto"},"mode":292}]}}

and checked that /toto was actually created at boot.
(I also tried some more advanced ignition config).

@tchernomax tchernomax requested a review from a team as a code owner September 26, 2025 16:28
@tormath1 tormath1 added the main label Sep 30, 2025
@tormath1 tormath1 moved this to ✅ Testing / in Review in Flatcar tactical, release planning, and roadmap Sep 30, 2025
@tormath1
Copy link
Contributor

Hi @tchernomax and thanks for your contribution, two questions:

  • About the naming, could we use only nutanix averywhere to match the OEM id? Or _ahv is required because multiple versions of Nutanix images might exist?

  • However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

    Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 30, 2025 •
edited
Loading

Build action triggered: https://github.com/flatcar/scripts/actions/runs/18348277688

@tchernomax
Copy link
Author 

* About the naming, could we use only `nutanix` averywhere to match the OEM id? Or `_ahv` is required because multiple versions of Nutanix images might exist?

I don't really know.
I know that nutanix provide an interface which can be plugged on top off VMWare ESX hypervisors (but in that case we use the vmware flatcar image).
There is also nutanix AHV which use a nutanix branded hypervisor (what this MR is for).
While googling I also found nutanix cloud platform and nutanix kubernetes platform but I don't really what it is.

I think the best curse of action would be to ask nutanix themself.
Tomorrow I will ask a colleague (I don't have access myself) to create an issue on nutanix support to ask them to reply on this PR.

(if there is no need for _ahv I will update this PR)

* > However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.
  Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

I didn't find any source online.
And the content of LICENSE file on hosts where the software is already install suggest it's not.

@aurelienlm
Copy link

I’ve asked Nutanix Support to take a look at this PR and provide feedback, just in case it helps

@aurelienlm
Copy link

the answer :
If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource.
2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

@tormath1
Copy link
Contributor

tormath1 commented Oct 7, 2025

the answer : If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource. 2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

Great thanks for asking. That'd be nice to have those guest tools to get the full support. Do you know what they add? I guess it's some helpers for network, system information, etc. ?

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

@tchernomax
Copy link
Author

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine.
(though we only tests flatcar on nutanix (without tools) on a simple POC)

@tormath1
Copy link
Contributor

tormath1 commented Oct 7, 2025

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine. (though we only tests flatcar on nutanix (without tools) on a simple POC)

Ok then, we can start with an image without guest tools and revisit this later.

To get back on the first point (nutanix_ahv vs nutanix) that'd be nice to know which one we can use, to have some consistency with the OEM ID. Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

@aurelienlm
Copy link

sorry the Nutanix support team can't help us on this. They have close the case.

I have reach out to our account team

@tchernomax
Copy link
Author

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

@tormath1
Copy link
Contributor

tormath1 commented Oct 8, 2025

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

Thanks! I'll setup our CI system to build nutanix images then. Once the image will be built, will you be available for running test for it?

@tchernomax
Copy link
Author

Once the image will be built, will you be available for running test for it?

yes (I might not be very responsive but I will make time to tests the images you provide)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

main

Projects

Flatcar tactical, release planning, and roadmap
Status: ✅ Testing / in Review

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tchernomax @tormath1 @aurelienlm